1. Field of the Invention
The present invention relates to a recognition device and a recognition method for recognizing a three-dimensional position and orientation of an article, which are used when the article is taken out by using a robot.
2. Description of the Related Art
In a robot system for taking out an article by using a robot, information regarding the article is obtained by a camera or a three-dimensional vision sensor connected to an information processor or a robot controller, and the obtained information is used to correct the motion of the robot which takes out the article or inspects the article.
As the three-dimensional vision sensor, for example, various range sensors, capable of measuring a distance to an object to be measured, are well known. Some range sensors can obtain three-dimensional information of a relatively wide area. These range sensors may obtain information regarding a three-dimensional shape of an article, which cannot be obtained by an imager such as a camera, as a range imagery or a height map or the like.
Japanese Unexamined Patent Publication (Kokai) No. 7-270137 discloses a spotlight scanning-type three-dimensional vision sensor, and describes that “the three-dimensional vision sensor makes it possible to effectively carry out flexible measurement, corresponding to the size, shape, perspective and required three-dimensional positional information of the object to be measured, by constituting a measurement system wherein a spotlight injection means capable of freely changing the light injecting direction by two-dimensional random scanning control is combined with a position sensing detector (PSD) having a one-dimension position sensing function.”
Japanese Unexamined Patent Publication (Kokai) No. 9-196636 discloses a method for obtaining data of the shape of an article, and describes that “in carrying out calibration of a position detector 1 and obtaining a shape of an article by using position detector 1, which is constituted by a slit light source 2 for irradiating slit light I to article W to be measured and a camera 3 for capturing am image of slit light I, position detector 1 is moved at predetermined times so as to determine a camera parameter in a perspective transformation equation for converting a coordinate system of article W into a coordinate system on the image by camera 3, a lit light parameter in a plane equation of slit light I irradiated on article W, and a model parameter in a geometric representation equation for representing the shape of article W. Then, the calibration of the slit light and the camera and obtaining of the shape data of article W are automatically carried out together in the robot coordinate system.”
Japanese Unexamined Patent Publication (Kokai) No. 2005-62063 describes that “an irradiation pattern is irradiated onto an article to be measured by means of a projector 2, a reflective image pattern of the irradiation pattern on article 1 is captured by means of a camera 8, the surface shape of article 1 is calculated by means of a shape calculating part 13 based on the reflective pattern, and a contrast ratio representing the measurement reliability of the article is calculated based on (Ia/(Pub−Plb)) by means of a reliability calculating part 14, wherein Plb and Pub are lower and upper limits of a light volume of the irradiation pattern, respectively, and Ia is the light volume. Then, data of the surface shape and the contrast ratio is output by a data outputting means, whereby the reliability of the shape measurement data of article 1, which is subject to multiple reflection, may be precisely detected corresponding to various pattern projection methods and correctly determined the relationship to each pixel of the captured image.”
Further, Japanese Unexamined Patent Publication (Kokai) No. 2004-272841 discloses an article discriminating device having a scanning means for scanning a laser pulse in the width direction of a road, a reflected light detecting means for detecting reflected light of the laser pulse when the scanning means scans the laser pulse in the width direction of the road, a discriminating means for obtaining the height and width of an article positioned on a scanning line of the laser pulse obtained by the scanning means, and discriminating the type of article, by using a time lag between a timing when the scanning means scans the road in the width direction by using the laser pulse and a timing when the reflected light detecting means detects the reflected light of the laser pulse scanned in the width direction of the road, wherein the discriminating means judges that the article is a pedestrian when the height and width of the detected object in a pedestrian detection area having a certain width arranged at a side of the road are within a predetermined range.
On the other hand, some methods have been proposed for recognizing the position and orientation of an article by using a range sensor (or a distance sensor). For example, Japanese Unexamined Patent Publication (Kokai) No. 2009-128191 discloses an article recognition device and a robot device, and describes that “according to the article recognition device of the invention, by carrying out high-speed positioning based on an amount of characteristic such as a spin image, three-dimensional shape data (model) of an object and distance data (scene) obtained by a range sensor may be compared, whereby three-dimensional position and orientation of the object may be rapidly recognized.”
Japanese Unexamined Patent Publication (Kokai) No. 2004-144557 discloses a three-dimensional vision sensor and describes that “sensor body 110 is constituted by light projector 130 attached to the hand of robot 40 and video camera 30. Light projector 130 projects slit light 131 onto a surface of object W positioned in operation area 50, video camera 30 receives a reflected light, and image processor 2 analyzes the reflected light and determines the surface where slit light 131 exists. Then, video camera 30 obtains an image by normal imaging, and a view line passing through measurement point Q is determined by using calibration data.”
Further, some methods, for detecting the position of an article from an image, have been proposed. For example, Japanese Unexamined Patent Publication (Kokai) No. 6-165036 discloses a method for aligning radiological images, and describes that “template regions 8, 8′ are set on X-ray image 4a, and template regions 9, 9′ are set on X-ray image 4b. Then, template matching is carried out for aligning template regions 8, 8′ with template regions 9, 9′ by using a correlation method.”
As a typical method for recognizing the position and orientation of an article by using a range sensor (distance sensor), a method for matching three dimensional information obtained by the range sensor with three-dimensional shape data of the article is possible, as disclosed in Japanese Unexamined Patent Publication (Kokai) No. 2009-128191. Generally, the conventional recognition method has very high cost of calculation, and requires a high-performance processor and a large-capacity memory. Therefore, it is very difficult to carry out such recognition process in a short period of time. Further, it is necessary to recognize the article with accuracy in order to take out the article by using the robot, however, a spatial density of the three-dimensional information obtained by the range sensor must be high in this case. Nevertheless, when the spatial density of the three-dimensional information is increased, a measurement time of the range sensor is extended and a calculation cost for matching process regarding the three-dimensional shape data of the article is also increased. Therefore, the taking out operation cannot be carried out.
The sensor for determining the three-dimensional information based on a time-of-flight of the laser pulse, as described in Japanese Unexamined Patent Publication (Kokai) No. 2004-272841, generally has a large size, and thus a cheaper sensor cannot be provided. Therefore, at this time, the range sensor has not been put into practical use in the field of the taking out operation using the robot.
On the other hand, although a method wherein the three-dimensional sensor and the two-dimensional sensor are combined, as described in Japanese Unexamined Patent Publication (Kokai) No. 2004-144557, is not intended to used the range sensor, the method is capable of recognizing the three-dimensional position and orientation of the article with relatively low calculation cost and amount of memory in comparison to the above recognition process. However, while the method in Japanese Unexamined Patent Publication (Kokai) No. 2004-144557 may reduce the calculation cost for processing the three-dimensional information, the method requires the process as described in Japanese Unexamined Patent Publication (Kokai) No. 6-165036, wherein the article is found within a measurement range of the sensor by using the two-dimensional image, and the calculation cost for the latter process may be increased.
It could be understood that the method of Japanese Unexamined Patent Publication (Kokai) No. 6-165036 is to recognize the two-dimensional positions of the articles having the different heights or orientations, based on the two-dimensional image.
However, in the method, it is necessary to carry out template-matching while enlarging or reducing the template in various sizes. Therefore, such a process takes time and it is difficult to realize the practical taking out operation.